目的：利用机器学习方法，我们的目标是在患者报告的糖尿病相关的推文中提取明确和隐含的造成关联，并提供一种更好地了解糖尿病在线社区内共享的意见，感受和观察的工具，从而从因果关系角度来。材料和方法：2017年4月至1月2021年间收集了3000多万糖尿病英语糖尿病相关推文。应用深度学习和自然语言处理方法，专注于具有个人和情感内容的推文。将一个原因效果 - Tweet数据集手动标记并用于训练1）微调BERTWEET模型，以检测包含因果关系2）的因果句，其中基于BERT的特征，以提取可能的原因效果关联。以半监督方法聚类原因和效果，并在交互式原因效果网络中可视化。结果：在不平衡数据集中的召回中检测到因果句，召回68％。具有基于BERT的特征的CRF模型表现出用于效果检测的微调伯特模型，具有68％的宏观召回。这导致了96,676个句子与原因效应关联。 “糖尿病”被鉴定为中央簇，然后被“死亡”和“胰岛素”。胰岛素定价相关原因经常与“死亡”相关。结论：开发了一种新颖的方法来检测因果句，并确定与糖尿病相关推文中的显式和隐含，单词和多字原因和相应的效果，利用基于伯伯的架构，并被视为原因效果网络。提取现实生活中的因果关系，患者报告社交媒体数据的结果提供了糖尿病研究中有用的互补信息来源。

Many machine learning problems encode their data as a matrix with a possibly very large number of rows and columns. In several applications like neuroscience, image compression or deep reinforcement learning, the principal subspace of such a matrix provides a useful, low-dimensional representation of individual data. Here, we are interested in determining the $d$-dimensional principal subspace of a given matrix from sample entries, i.e. from small random submatrices. Although a number of sample-based methods exist for this problem (e.g. Oja's rule \citep{oja1982simplified}), these assume access to full columns of the matrix or particular matrix structure such as symmetry and cannot be combined as-is with neural networks \citep{baldi1989neural}. In this paper, we derive an algorithm that learns a principal subspace from sample entries, can be applied when the approximate subspace is represented by a neural network, and hence can be scaled to datasets with an effectively infinite number of rows and columns. Our method consists in defining a loss function whose minimizer is the desired principal subspace, and constructing a gradient estimate of this loss whose bias can be controlled. We complement our theoretical analysis with a series of experiments on synthetic matrices, the MNIST dataset \citep{lecun2010mnist} and the reinforcement learning domain PuddleWorld \citep{sutton1995generalization} demonstrating the usefulness of our approach.

We study the learning dynamics of self-predictive learning for reinforcement learning, a family of algorithms that learn representations by minimizing the prediction error of their own future latent representations. Despite its recent empirical success, such algorithms have an apparent defect: trivial representations (such as constants) minimize the prediction error, yet it is obviously undesirable to converge to such solutions. Our central insight is that careful designs of the optimization dynamics are critical to learning meaningful representations. We identify that a faster paced optimization of the predictor and semi-gradient updates on the representation, are crucial to preventing the representation collapse. Then in an idealized setup, we show self-predictive learning dynamics carries out spectral decomposition on the state transition matrix, effectively capturing information of the transition dynamics. Building on the theoretical insights, we propose bidirectional self-predictive learning, a novel self-predictive algorithm that learns two representations simultaneously. We examine the robustness of our theoretical insights with a number of small-scale experiments and showcase the promise of the novel representation learning algorithm with large-scale experiments.

使用现实世界数据的背景临床研究可能会受益于利用临床报告，这是一种特别丰富的非结构化培养基。为此，自然语言处理可以提取相关信息。使用预训练的语言模型基于转移学习的方法已在大多数NLP应用程序中实现了最先进的方法；但是，公开可用的模型缺乏接触专业语言，尤其是在医学领域。目标我们旨在评估将语言模型适应法国临床报告对下游医疗NLP任务的影响。方法我们利用从2017年8月至2021年7月在大巴黎大学医院（APHP）收集的2100万临床报告的语料库，以生产两种有关专业语言的卡梅蒙德体系结构：一项从Scratch中进行了再培训，另一个以Cammembert作为其初始化。我们使用两个法国注释的医学数据集将我们的语言模型与原始的Camembert网络进行比较，从而评估了Wilcoxon测试改进的统计意义。结果我们在临床报告上预估计的模型将APMED（APHP特定任务）的平均F1分数提高了3个百分点，达到91％，这是统计学上显着的改善。他们还达到了与Quaero上的原始Camembert相当的性能。这些结果适用于很少的预训练样品开始，从而对微调和划痕版本构成了这些结果。结论我们确认以前的文献表明，适应通才培训的语言模型（例如Camenbert on Specialty Corpora）改善了其下游临床NLP任务的性能。我们的结果表明，与微调相比，从头开始进行重新培训不会引起统计学上显着的性能增长。